Data flow programming may involve developing an application that is constructed as computational “nodes” that execute user or predefined functions. These nodes may be connected by “edges” that represent the communication channels between the computations. As messages arrive at a node, the node may apply its function and then may forward the result of the function to other nodes that may be connected to it by its out-going edges. If all data dependencies are expressed using edges, the parallelism in the application may become simple to infer and a runtime system may use the parallelism to exploit the computational resources available on a platform, whether they may be cores in a multiprocessor or systems in a cluster.
When developing a data flow application, developers may make decisions about the structure of the flow graph. For example, the developers may choose what code should be encapsulated in each node and what dependencies may be expressed as edges. In distributed memory implementations, decisions may be made about placement on devices or data transfers across memory domains. Additional properties may also be able to be set on the nodes and edges. There may be many ways to express the same algorithm as a data flow graph. The choice of graph structure, placement and node properties may impact the resulting performance and scalability of the parallel execution, and therefore iterative tuning of the graph structure may be needed to obtain optimal performance.